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Abstract 

Both publicly and privately funded research projects managed by universities are growing in 
size and scope. Complex, large-scale projects (over $50 million) pose new management 
challenges and risks for universities. This paper explores the relationship between project 
success and a variety of factors in large-scale university projects. First, we characterize the 
challenge of large-scale university research project management, synthesize findings from the 
literature, and identify research gaps. Second, we offer a profile of large-scale U.S. university- 
run projects based on a survey conducted by us. The projects address a range of research from 
large-scale clinical trials to the construction of complex scientific instruments. While NIH is the 
largest overall government funder of university research, NASA is the largest funder of these 
large-scale university projects. Third, we share some preliminary results from our survey. While 
universities often meet their technical performance goals, cost and schedule overruns are 
common and can be significant. Qualitative data confirmed that research project managers face 
anti-management challenges in the university setting and challenges with project management 
techniques not tailored to the university. 


Introduction 

Th e number of large, complex research 

projects managed by universities is growing 
in size and scope. Additionally, industries 
are shifting more research projects to 
universities (Hall, Link, & Scott, 2003). At 
Arizona State University (ASU), for 


example, the amount of annual funding for 
sponsored projects over $5 million has risen 
from $10 million to $40 million over the past 
twenty years (Raudenbush, 2011). Large- 
scale research projects (over $50 million) 
pose management challenges and risks 
because they are often complex and 
unpredictable, involve new technologies. 
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involve a large number of stakeholders and 
institutions, and extend over a long time 
scale (Bonnal, Jonghe & Ferguson, 2006). 
The traditional project management (PM) 
literature offers a number of methodologies 
for managing risk and uncertainty in large- 
scale projects, with a focus on minimizing 
cost and schedule overruns. However, 
universities are not known for the 
implementation of sophisticated project 
management systems and the management 
methodologies may fall by the wayside of 
'getting the science right.' The relatively 
new field of research project management 
(RPM) is still developing its professional 
identity and gaining legitimacy, and 
scientists have resisted managers' attempts 
to engage in research project management 
(Sapienza, 2004; Schuetzenmeister, 2010). 

As the scope and size of research 
projects expand, universities have become 
major players, and sometimes leaders, in 
multi-million dollar research and 
development (R&D) projects. For example, 
in 2009, the National Science Foundation 
(NSF) awarded over $200 million to the 
University of Wisconsin-Madison to 
construct a deep-ice neutrino detector, 
called IceCube, in Antarctica. In 2007, the 
National Institutes of Health (NIH) 
awarded over $63 million over three years 
to George Washington University to 
develop a Diabetes Prevention Program. 
Over the past decade, NASA has granted 
several dozen awards over $100 million to 


universities for first-of-a-kind spacecraft 
research and development, including 
Genesis, Deep Impact, and Galex. These 
research projects are often: (1) extremely 
complex; and (2) decentralized, with work 
occurring at multiple institutions and across 
disciplines; and (3) may change 
dramatically in scope. All of these risk 
factors contribute to cost and schedule 
overruns in large-scale projects. 

.. large-scale research project 
management techniques should 
be improved in order to increase 
project success." 

This paper explores the current 
population of completed or nearly 
completed large-scale university-run 
research projects and then examines the 
relationship between university 
management techniques and project 
success. We demonstrate that a significant 
amount of funding is spent on university- 
led projects larger than $50 million and 
argue that large-scale research project 
management techniques should be 
improved in order to increase project 
success. We begin by discussing how large- 
scale projects are defined and characterized 
in the literature and how we apply these 
characteristics to large-scale university-run 
research projects. Next, we discuss a 
number of challenges facing university 
research project managers. We then offer an 
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overview of project outcome measures, 
primarily cost, schedule, and technical 
performance, and discuss how these apply 
specifically to R&D. Next, we describe our 
methodology for developing a sample of 
large-scale university-run projects, which 
we believe is the total population minus 
U.S. Department of Defense projects. We 
describe the attributes of this population, 
finding a median university total project 
cost expenditure of $93,586,025 and an 
average project timescale of seven years. 
Finally, we share some preliminary findings 
from our survey of managers of these 
projects. While many university projects 
meet their overall technical objectives, many 
do so by overrunning the original cost and 
by slipping the initial schedule. 

What is a Large-Scale Research 
Project? 

Th e large-scale project and its even 

larger counterpart, the megaproject, are 
often defined based on cost ranges, though 
these cost ranges vary throughout the 
literature. For example, Flyvbjerg (2007) 
defined large-scale projects as those that 
cost between $100 million and several 
billion dollars. He defined megaprojects as 
projects over $1 billion with a lifetime of 50 
years or more (Flyvbjerg, 2005). Merrow 
(1988) defined large-scale projects as those 
over $500 million, and he defined projects 
over $1 billion as "very large projects." 
Large-scale research projects, however, are 


generally lower in cost than the large-scale 
infrastructure projects on which much of 
the literature on large-scale projects focuses. 
Further, projects on which the university is 
the lead manager are typically on the lower 
end of the large-scale research project cost 
range. 

For the purposes of this study, we used 
a cost threshold of over $50 million in total 
project costs to characterize large-scale 
research 1 projects, of which there are 
roughly 58 U.S. university-led projects in 
the United States (as of 2010). 2 This choice 
was empirically driven. If we had used the 
$100 million cut-off from the literature 
described above, the sample would have 
been limited to 21 projects and skewed 
toward National Aeronautics and Space 
Administration (NASA) and U.S. 
Department of Energy (DOE) projects led 
by California Institute of Technology 
(Caltech) and Stanford University. Types of 
scientific and technical research projects 
over $50 million include the construction of 
complex scientific instrumentation; the 
construction of first-of-a-kind spacecraft; 
the design of innovative weapons systems; 
the construction of large-scale, first-of-a- 
kind computing infrastructure; longitudinal 
clinical trials; and bioscience research 
projects with a singular objective. Due to 
our focus on university management, we 
did not address scientific megaprojects, 
which are typically over $1 billion and 
international in scope, and extend over 
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decades (Cross, 2009). Scientific 
megaprojects are often managed by one or 
more government agencies rather than 
universities. Examples of megaprojects 
include the International Space Station, the 
Human Genome Project ($3 billion), and the 
Superconducting Super Collider (expected 
cost of $8 billion, but cancelled in 1993). 
While these "Big Science" projects garner 
much attention, guidance and program 
evaluation are also needed on middle-range 
university-led projects, which also represent 
significant research expenditure. 

We defined a project as having a clear 
objective and timescale and a definable 
outcome. In contrast, many large-budget 
scientific operations are research programs 
with components at multiple universities, 
across scientific domains, and sometimes 
representing several countries. Basic 
research at this funding scale tends to be 
conducted at dozens of universities 
working to fulfill a research center's 
typically broad mission. NIH funds a 
number of research programs distributed 
across multiple universities, such as the 
Center for AIDS research, national/regional 
primate research centers, and the general 
clinical research center. These did not fall 
under our definition of a project. 

Basing the definition on a range of total 
project costs is likely not the only, and 
perhaps not the most useful, metric for 
characterizing these large-scale research 
projects in a way that facilitates 


understanding of the management 
challenges they pose. Large-scale research 
projects are also often highly complex and 
uncertain, extend over long time scales, and 
involve a large number of stakeholders and 
researchers. Bonnal et al. (2006) 
characterized large-scale projects based 
upon the following factors: number of 
contributors to the project; number of 
activities the project seeks to perform and 
the relative complexity of these activities; 
number of intermediate deliverables that 
are produced throughout the project's 
execution; number of activities outsourced 
to external contractors; and project duration 
that can span over a decade, making it 
difficult to define the long-term objectives 
of the project at the project's conception. In 
fact, these characteristics are more relevant 
to research projects, which may have 
smaller budgets than construction projects 
but be extremely complex in terms of the 
number of involved actors and institutions; 
the number of experiments and activities; 
the long time periods, particularly until the 
science is translated into a societal benefit; 
and the number of stakeholders ranging 
from human subjects to the policymakers 
and taxpayers funding the research. In 
summary, large-scale research projects, for 
the purposes of this study, have a clear and 
achievable research goal, are very complex, 
often involving uncertain technologies, 
typically involve a large number of actors 
and institutions, often involve relatively 



Research Management Review, Volume 18, Number 2 

Fall/Winter 2011 


long time scales, and cost over $50 million 
in total project costs. 


The Challenges of Managing 
Large Projects at Universities: 
Management Knowledge and 
the Research Management 
Profession 


Universities are being called upon to 


manage increasingly large research and 
technology development projects, but there 
has been a surprisingly small subsequent 
gain in the systematic knowledge of the 
challenges and risks involved with 
university research project management 
(RPM). Universities face challenges in each 
stage of managing large-scale projects: 
winning the project, defining the project, 
and managing the project. Universities face 
two key conceptual issues when managing 
large-scale projects. First, universities are 
not designed as project management 
organizations and therefore are not 
necessarily equipped to manage these 
behemoth projects in an efficient manner. 
Second, research does not progress in a 
linear fashion in the way that construction 
projects often do. Research project 
managers face the discovery paradox, 
meaning that discovery occurs in 
serendipitous ways, 3 but existing 
management techniques are typically linear 
and prescribed. 


To address these challenges, 1) 
knowledge of university project 
management is needed, and 2) experienced 
research project managers are needed. We 
will address the need for RPM knowledge 
first. The traditional project management 
profession has developed a set of project 
management tools based initially on 
experience with construction and 
infrastructure projects. These tools have 
been refined for large-scale weapons 
systems, environmental clean-up and 
restoration projects, and large-scale 
information technology projects. This 
experience and research have even been 
synthesized in a number of publications, 
including the Project Management Body of 
Knowledge published by the Project 
Management Institute. However, higher 
education institutions are often the slowest 
adopters of project management (Kralevich, 
2008). While there is some synergy with 
traditional project management techniques 
and research project management, PM 
techniques designed specifically for 
university are lacking (Austin, 2002; Erno- 
Kjolhede, Husted, Monsted, & Wenneberg, 
2001; Powers & Kerr, 2009). This is a gap in 
need of further research. 

As mentioned above, these research 
project managers face the scientific 
discovery paradox. Geles et al. (2000) 
argued that most project management 
strategies were designed for business, not 
science; further, this literature is not 
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"Research project managers face 
the discovery paradox, meaning 
that discovery occurs in 
serendipitous ways, but existing 
management techniques are 
typically linear and prescribed." 

rigorous. Geles et al. outlined some of the 
project management strategies they believe 
would be suitable for use in the laboratory, 
including using a work breakdown 
structure for planning, charting the overall 
resource inputs required for the project, 
planning for risks and contingency, 
scheduling using Gantt Charts and project 
milestones, and using a costing scheme that 
converts resources into a common unit, e.g., 
U.S. dollars. Others feel more strongly that 
a completely new set of methods should be 
developed. Austin (2002) argued that the 
project management literature is too 
uniform and is not adaptive enough to be 
applied to science where there is "genuine 
discovery" that cannot be anticipated and 
planned for. Conventional project 
management strategies are better suited for 
construction because it is more predictable. 
Instead of spending a lot of time planning, 
research projects will require some 
learning-by-doing. Therefore, Austin 
argued, dynamic research project 
management methods with adaptive 
approaches are needed for innovative 
projects. This discovery paradox is a key 


challenge facing research project managers 
moving forward. 

Also needed are experienced research 
project managers. A National Research 
Council report argued that Ph.D. scientists 
have not been trained in management, so 
large-scale research projects will require a 
research project manager who should be 
hired based on management skills, not 
scientific credentials (Nass & Stillman, 

2003). These research project managers 
fulfill an important role in managing a 
growing amount of external funding for the 
university and their level of experience is 
thought to contribute to keeping projects on 
schedule and budget. This unique 
profession interweaves academic, 
managerial, and public service training and 
skills (Schuetzenmeister, 2010). It works at 
the boundary between science and society, 
managing and negotiating with multiple 
stakeholders both in and outside of the 
academy (ibid.). 

.. most project management 
strategies were designed for 
business, not science ... 

However, the research management 
profession is still forming its professional 
identity, struggling to delineate itself from 
university administration (ibid.). It is also 
working to prove that it possesses 
legitimate expertise and a credible identity 
in laboratories that have traditionally seen 
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themselves as self-governing. There is a 
stigma that managing science is not as good 
as doing science, and large-scale research 
project managers must work to develop an 
interactional form of expertise even though 
they are often not trained in the particular 
field of study and are often less eminent 
than the scientists they are managing 
(Collins & Sanders, 2007). Research project 
managers face the following challenges in 
the lab: forces that pull research teams 
apart, anti-management (i.e., resistance to 
being managed), goal conflict between 
scientists and managers, difficulty with 
performance evaluation, and anti¬ 
organization because following the 
scientific method is viewed as providing 
sufficient organization (Smith & Tuttle, 
1988). Further, large-scale research projects 
often require management across multiple 
disciplines and communications barriers 
may increase the project risk (Sapienza, 
2004). 


"There is a stigma that managing 
science is not as good as doing 
science..." 


Managing Cost, Schedule, and 
Technical Risks in Large-Scale 
Research Projects 


All large-scale projects entail a unique 


set of management challenges. These 
include the following: the technology 


involved is often not standard, the decision¬ 
making process includes multiple actors 
with conflicting interests, the project scope 
and ambition level change over time, 
contingency estimates are usually 
inadequate despite statistical forecasts, and 
misinformation about costs and benefits is 
the norm (Flyvbjerg, 2005). These factors 
may result in cost overruns and 
performance shortfalls in a majority of 
projects (ibid.). Project management is 
geared toward increasing planning to 
reduce project uncertainty and risk. Project 
risk is a function of complexity, 
innovativeness, project definition, 
management experience, regulatory 
environment, budget certainty, and error 
(Cash, McFarlan, & McKenney, 1992; 
Merrow, 1988; Myers et al., 1986; Weil, 

1992). 

Schedule Slip 

Changing project scope and poor project 
definition are key contributors to schedule 
slippage. Defining the project includes 
mapping out the project tasks, task 
relationships, project environment, and 
outcomes. Poor project definition has been 
shown to be a key contributing factor to cost 
and schedule overruns (Myers et al., 1986). 
Myers et al. (1986) also found that slippage 
in the project's total startup time could be 
explained by the number of process steps 
that were not commercially proven and by 
dispersed project responsibility. It is also 
widely acknowledged in the literature that 
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project scope change is a major contributor 
to risk and uncertainty and drives schedule 
slippage. For instance, Samid (1994) found 
that one of the biggest challenges in R&D 
projects is that there are so many late- 
planned changes that the project bears little 
resemblance to the original plan (a 
reflection of the discovery paradox). 
Managing Cost Uncertainty and 
Overruns 

Cost growth, or cost escalation, is the 
difference between the estimated cost and 
the actual cost of the project (Merrow, 1988). 
In large-scale projects, technical goals 
usually take priority over time and cost 
goals (Grun, 2004). Therefore, cost growth 
in large government-funded construction 
and infrastructure projects has been a major 
focus in the PM literature. Unsurprisingly, 
cost escalation is identified as a major 
problem in these larger projects, with 
overruns of 50-100% being common 
(Skamris & Flyvbjerg, 1997). It seems that 
cost overruns are determined early in a 
project's lifespan; Christensen (1993) found 
that defense contracts are highly unlikely to 
recover from cost overruns incurred in the 
first 15% of the project. 

Cost overruns are also often blamed on 
mis-estimation. In one study of 
infrastructure projects, underestimation was 
found to occur in nine out of ten cases 
(Flyvbjerg, Bruzelius, & Rothengatter, 2003). 
Priemus et al. (2008) analyzed cost estimates 
in megaprojects and found that they have 


not improved in the past 70 years, and 
Ramachandran (1989) found that while cost¬ 
estimating methodology has become much 
more sophisticated, the level of accuracy 
has not improved. There is disagreement in 
the literature about the reasons for mis- 
estimation. Sometimes it is attributed to 
appraisal optimism. Samid (1994) found 
that in construction projects contingency is 
often just used to pad cost estimates, rather 
than being thoroughly analyzed. Bruzelius 
et al. (2002) found that in megaprojects of $1 
billion or more, the difference between the 
cost forecast and actual costs could not be 
attributed to inability to predict the future 
alone. They concluded that project 
proponents are intentionally biasing the 
forecasts, leading to poor decision-making 
by policymakers who are unable to 
rigorously evaluate the costs and benefits of 
a project because of these biased forecasts. 
They assert that technical error is actually a 
minor part of the cost overrun. Flyvbjerg 
(2007) recommended subjecting project 
forecasts for publicly funded projects to 
rigorous peer review. 

.. while cost estimating 
methodology has become much 
more sophisticated, the level of 
accuracy has not improved." 
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R&D and Risk 

One of the main risk factors addressed 
in the literature is technological complexity, 
also referred to as the level of innovation in 
the project or the use of 'unproven 
technologies' (Parker, Benson, & Trainor, 
1988; Sadeh, Dvir, & Shenhard, 2000; 
Shenhard & Dvir, 1996). The level of 
technological innovation in the project 
contributes to uncertainty and can result in 
cost escalation (Melamed, Skokan, 
Zenkowich, & Kocher, 2008; Merrow, 1988). 
The three measures of the level of 
innovation are whether 1) the project used a 
first-of-a-kind technology, 2) it employed 
new materials or methods, and 3) it was the 
largest project of its kind when it was 
constructed. 

R&D brings with it uncertainty that is 
difficult to quantify. Pinto and Covin (1989) 
drew distinctions between R&D projects 
and construction projects due in part to 
overt risk. Rigorous, yet flexible, techniques 
are needed (Samid, 1994). Previous 
knowledge is required for effective 
statistical analysis, yet in R&D projects the 
assumption that previous knowledge can be 
used to predict outcomes often fails (ibid.). 
Austin (2002) argued that risk management 
for highly uncertain R&D projects might 
need to be different from the risk and 
uncertainty methodology that has been well 
developed in the construction industry. In a 
survey about the usefulness of risk 
management strategies, Galway (2004) 


found that construction managers are 
wedded to risk management, but high 
technology practitioners are ambivalent 
toward it. 

Large-Scale University 
Research Project Management 
Survey Results 

Methodology 

w e developed a sample of recently 

completed or nearly complete research 
projects over $50 million in which a U.S. 
university was the primary leader. Our 
sample size was 58, which we believe is the 
total population minus U.S. Department of 
Defense (DoD) projects. We faced several 
challenges developing this sample. First, 
there was little freely available information 
on the number and manager of large-scale 
research projects funded in the United 
States. Secondf, grant money was often 
distributed over multiple years or even 
through multiple grants and thus difficult 
to aggregate. We obtained project lists from 
the NIH, Centers for Disease Control 
(CDC), NSF, DOE, and NASA. Only NSF 
hosts a publicly available online database 
that may be searched by project cost. NASA 
and DOE staff provided us with 
information from internal databases. DOE 
maintains an online research and 
development database, but it is not 
searchable by cost. The NIH hosts a publicly 
available database with grant information. 
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but it also cannot be searched by project 
cost. NIH and CDC required us to submit 
Freedom of Information Act (FOIA) 
requests in order to obtain the data. We 
were unable to obtain data from the DoD; 
the DoD officials we contacted were 
unaware of any DoD-funded university-run 
projects over $50 million. We also contacted 
the sponsored projects office at major 
universities, but most were unable to 
provide us with a list of their large-scale 
projects. 

We designed a 28-page survey 
instrument that addressed the 
characteristics of the project, information 
about the project manager and project team, 
whether the project experienced cost 
overruns or schedule slip, what factors 
contributed to a successful project, the 
management and planning techniques used, 
and demographics. We asked the 
respondents to report on their initial cost 
and schedule estimates and their final cost 
and schedule outcomes to correlate these 
outcomes with a variety of risk factors. The 
development of the questions was theory- 
driven, drawing on factors in the literature 
thought to contribute to project success. 

The survey was administered to project 
managers online through SurveyMonkey. If 
the project did not have an RPM or the RPM 
could not be reached, the survey was sent to 
the project's Principal Investigator. We sent 
an alert letter to both the university's office 
of research and to the head of the project 


prior to sending the survey invitation. We 
also sent multiple email requests and made 
follow-up calls aimed at boosting the 
response rate. 

Characteristics of the Project Sample 

Our development of the sample offers a 
unique overview of the characteristics of 
university-run large-scale research projects. 
The median university total project cost 
expenditure for the sample was $93,586,025. 
The average timescale for these projects was 
roughly seven years (or a median of 6). 4 The 
federal government is the main funder of 
projects of this magnitude. There was no 
systematic method for searching for state- 
funded projects, and only one was 
uncovered through internet and database 
searches. The sample consisted of three 
CDC projects, 11 DOE projects, 24 NASA 
projects, 15 NIH projects, six NSF projects, 
and one Ohio Department of Transportation 
project (see Figure 1). 

Even though NIH is the largest overall 
government funder of university research 
(AAAS, 2012), it does not fund the greatest 
number of large-scale university-run 
projects. This is because much of NIH's 
research funding is dispersed across 
universities and is not project-based. For 
instance, much of NIH's expenditures are 
spent on basic research that is expected to 
one day translate into societal outcomes. 
NASA is the biggest funder of university- 
run large-scale research projects. 
Government funding for large-scale 
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research projects consists of large NASA 
space contracts, complex large-scale 
scientific instrumentation and research 


using this instrumentation funded by DOE 
and NSF, and longitudinal clinical trials and 
other project-based biomedical research 


Agency funders of large-scale university 

research 



■ DOE 18% 

■ NASA 39% 

■ NIH 11% 

■ NSF 25% 

■ OH Dept, of Transport 2% 
■ CDC5% 


Figure 1 . Funding Agencies for Large-Scale University-Run Projects Based on the 
Number of Recently Completed or Almost Completed Projects (through 2010). 
Note that this describes the survey sample, not survey respondents. 


funded by NIH. 

Four universities stand out as leaders in 
winning large-scale project contracts: 
California Institute of Technology (six 
projects plus one in partnership with 
Colorado State University, one in 
partnership with Hampton University, two 
in partnership with the Southwest Research 
Institute, and two in partnership with the 
University of California, Los Angeles), MIT 
(three projects), Stanford University (five 
projects), and the University of California, 
Berkeley (three projects). 


As illustrated by Crow and Bozeman 
(2001), national laboratories provide 
leverage for universities to win large-scale 
projects. For example, Caltech's leadership 
may be attributed to its close partnership 
with JPL and its history of leadership in 
space projects. Many of the universities in 
the sample have partnerships with national 
laboratories, including Fermi National 
Laboratory (University of Chicago), 
Princeton Plasma Physics Laboratory 
(Princeton University), Jet Propulsion 
Laboratory (JPL) (California Institute of 
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Technology), and Lawrence Berkeley 
National Laboratory (University of 
California, Berkeley). Further, NASA 
provides an experienced project manager 
through one of its labs—e.g., NASA JPL, 
NASA Goddard, or NASA Langley—for all 
projects on which the university is the 
Principal Investigator. In other cases, a 
novel university hybrid organization 
managed the projects. For example, the 
construction of the FFigh-Performance 
Airborne Platform for Environmental 
Research (L1IAPER) was managed by a 
university research consortium, the 
University Corporation for Atmospheric 
Research. The project took five-years and 
was completed in 2006 for a total project 
cost of $80 million. It was funded by NSF. 

"... national laboratories 
provide leverage for universities 
to win large-scale projects." 

These projects were diverse, including 
construction of complex scientific 
instrumentation, construction of first-of-a- 
kind spacecraft, fundamental research, and 
clinical trials (see Figure 2). Twenty-four of 
the projects involved first-of-a-kind space 
missions. For example. Deep Impact was a 
mission to impact and take samples of a 
comet. It was headed by University of 
Maryland (providing the Principal 
Investigator and scientific team) with the 
JPL (providing the project management) 


and Ball Aerospace & Technologies 
Corporation (providing the flight 
hardware). The construction and mission 
took six years at a total project cost of $330 
million, $66 million of which went to the 
university. In another example, the NASA 
Wide-Field Infrared Survey Explorer 
(WISE) mission mapped the sky at four 
different infrared wavelengths with greater 
sensitivity than past maps. It was a $320 
million mission in total project costs—with 
$220 million going to Caltech—over 11 
years. The Principal Investigator was 
provided through the University of 
California, Los Angeles; NASA's JPL 
provided the project manager. 

The NIH-funded biomedical projects 
ranged from clinical trials to genetic 
sequencing. One example is the $156 million 
Health and Retirement Study by the 
University of Michigan—a longitudinal in- 
depth interview study of senior citizens 
living in the United States. The researchers 
are interviewing 22,000 Americans over age 
50 every two years. Another example is the 
sequence of the yeast genome by Stanford 
University at a total project cost of $97.5 
million. A third is the BARI II trial at the 
University of Pittsburgh, which was a 
multi-country clinical trial on type II 
diabetes and coronary artery disease 
funded at a total project cost of $55 million. 

NSF and DOE funded a number of 
complex scientific instrumentation 
construction projects. For example, the Ice 
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Cube project, run by the University of 
Wisconsin-Madison and funded by NSF, 
entailed the construction of a deep ice 
neutrino detector at the South Pole. The 
project required drilling 86 holes and 
installing 5,160 sensors for a total 
expenditure of $200 million. Another 
example is the Earthscope project funded by 
NSF and managed by Stanford University. 

It consists of 400 portable seismometers 
covering the entire United States, global 
positioning instruments positioned to 


observe fault zones in North America, and 
strainmeter instruments for observing and 
studying plate boundary processes and 
volcanic events. Other projects in the 
sample were large-scale research projects 
conducted on recently constructed complex 
scientific instrumentation such as the 
Alcator C-Mod Fusion Research Program 
funded by the DOE, the National Compact 
Stellarator Experiment, and the Stanford 
Linear Collider Research and Development. 


Type of project 



■ Basic and applied research 

■ Construction of complex 
scientific instrumentation 

■ First-of-a-kind spacecraft 

■ Biomedical research 

■ Other technology 
development 


Figure 2. Type of Project Based on the Number of Recently Completed or 
Almost Completed Projects (through 2010). As categorized by the authors. 


Survey Results 

We received 18 partial responses and 12 
complete responses from a sample of 58. 
Unfortunately, this was not a high enough 
response rate for statistical significance, but 
we were able to make some observations. 


outlined below. The dependent variable 
was project success, defined by whether the 
project met technical performance, cost, and 
schedule goals. Independent variables 
included key factors that the literature 
suggested would drive cost and schedule 
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overruns, such as years of experience of the 
project manager; adequacy of project 
planning, particularly cost and risk 
estimation methods; changes in the scope of 
the project; inadequate project definition; 
sufficiency of the cost estimate; and 
interdisciplinary communication barriers. 

Project success, defined as meeting 
technical performance, schedule, and cost 
goals, was mixed. The response rate for the 
cost and schedule slippage questions was 
low. Only project managers who met their 
technical performance goals responded to 
these questions, biasing the results toward 
successful projects. The response rate for the 
project cost estimate and actual expenditure 
was particularly low (n=5), perhaps due to 
the sensitivity of the question. Only one 
project manager reported meeting the 
project budget, while two experienced 
moderate overruns (5-10% of the estimate) 


and one experienced a significant overrun 
(15% of the estimate, or $15 million). Forty- 
two percent of projects came in on time or 
ahead, 29% were somewhat behind (i.e., 25- 
50% over schedule), and 29% were very 
behind (i.e., 25-50% over schedule) (n= 8). 
Therefore, while all of the projects delivered 
on their technical promises, many did so 
well over schedule. Figure 3 outlines project 
success, which we defined based upon the 
significance of the cost overrun, the 
significance of the schedule overrun, and 
whether technical performance goals were 
met. Sixty-seven percent of projects were 
somewhat successful, 11% were successful, 
and 22% were very successful. 5 Surpris¬ 
ingly, only half of the projects (n=18) 
conducted a project risk assessment in the 
planning phases, suggesting planning for 
risk and contingency was insufficient. 



Project_Success 


Figure 3. Project Success, as Defined by Cost, Schedule, and 
Technical Performance Goals. Source: PASW 
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Based on findings from the literature, 
the project manager's experience level was 
expected to affect project success. The 
majority of all managers who responded 
were highly experienced, with about 85% 
having five or more years of experience on 
projects over $50 million. Sixty-four percent 
had master's degrees in science, technology. 


or medicine, and 22% had Ph.D.s. (see 
Figure 4). Additionally, 61% had training in 
the scientific sub-discipline related to the 
project. In 77.8% of cases, project managers 
reported that the experience of the project 
staff was also critical to meeting technical 
performance goals. 



Project manager's education in science, technology or medicine 


Figure 4. Project Managers' Education in Science, Technology, or Medicine 
Source: PASW 


Fifty-seven percent of the projects 
experienced turnover in the lead project 
manager during the project. While we 
expected to find that turnover in the lead 
project manager negatively affected the 
project, two managers stated that the 
change was positive because the initial 
project manager was either inexperienced in 
large-scale project management or was 
inexperienced in the scientific domain. 
Several others stated that there was a 


negative impact at the time of the change, 
but overall the change turned out to be 
positive, or even very positive, because they 
gained a more experienced manager. 

The project managers' qualitative 
responses revealed challenges with anti¬ 
management and university-specific 
management techniques, reinforcing the 
findings from the literature on research 
management outlined above. One of the 
project managers reported that many of the 
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PM techniques NASA suggested they use 
were irrelevant to a university setting. 
Another pointed out that university clocks 
operate on different schedules than those of 
the aerospace contractors. Several managers 
reported that 'managing by walking around 
and speaking informally with people' was 
the most important technique. S/he stated, 
"people working on the mission need to 
know that you know them and that their 
contribution is important." In summary, 
project communication is important to 
success. 

Five shared frank comments about their 
experiences with anti-management, with 
one stating: 

. . . many of the individuals assigned 
to work on the project were 
unfamiliar with, and resistant to, the 
implementation of formal project 
management processes. This 
resistance often led to a hesitance 
(and in some cases a refusal) to work 
with me and others on the project 
team to perform appropriate cost, 
schedule, and status reporting. 

This finding reinforces the findings from 
the literature that research project managers 
face significant anti-management challenges 
and adds to it the possibility that these 
challenges may lead to cost and schedule 
overruns. One RPM reported significant 
staff turnover in the project due to anti¬ 
management. 


Other RPMs reported struggling with 
scientists who believed management and 
science were mutually contradictory, with 
one stating that: 

Some scientists within the 
organization firmly believed that 
management of the project was 
contrary to scientific discovery; that 
managing to a budget, schedule, and 
scope were 'anti-science.' 

This experience reflects the scientific 
discovery paradox. 

Discussion and Conclusions 

hese preliminary findings suggest 

that there is much more to be learned about 
managing university projects. Research 
managers will continue to face challenges 
with anti-management, the discovery 
paradox, and university design. Developing 
project management methods that are 
tailored to the university setting and to risk 
factors specific to large-scale research 
projects is necessary for moving universities 
toward even greater success in completing 
research projects on time and on budget. 

We discovered throughout the research 
process that federal agencies and 
universities lack data and data transparency 
about their large-scale projects. This makes 
it difficult to systematically develop a 
profile of the large-scale university-run 
research funded by the U.S. government. It 
also suggests a lack of coordination in this 
research profile. Additionally, most major 
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research universities were unable to 
provide us with data, with one sponsored 
projects office lamenting that such data 
were very difficult to collect in their 
decentralized institution. The NSF has taken 
an excellent first step with its online 
database of projects searchable by cost. 
Other agencies may consider developing 
such databases, adding cost parameters to 
their existing databases, or making data 
available through databases like data.gov. 


"Developing project 
management methods that are 
tailored to the university setting 
and to risk factors specific to 
large-scale research projects is 
necessary for moving universities 
toward even greater success in 
completing research projects on 
time and on budget." 


While rigorous project evaluation may 
improve future project success rates, it is 
difficult to execute. Barriers to effective 
project evaluation include: difficulty 
tracking evolving projects; concern over 
disclosure of proprietary information; and a 
lack of incentive for managers to 
participate, particularly if the project was 


unsuccessful (Galway, 2004). While the key 
objective of our survey was to determine 
the significance of cost and schedule 
overruns in these projects, the response rate 
in that section of the survey was 
particularly low. Further, several project 
managers reported to us that they were not 
allowed to participate in such a survey. The 
length of the survey also contributed to the 
low response rate, particularly since there 
are few incentives for extremely busy 
managers to devote time to program 
evaluation. The research management 
profession and funders should consider 
counteracting these barriers, perhaps with 
incentives offered for participating in 
program evaluation. As management 
methodologies improve, research project 
managers will likely benefit from focusing 
on and improving project definition and 
structuring projects to reduce their 
complexity. Conducting a project risk 
assessment at the initiation of the project is 
also likely to aid in success. Universities 
may also consider opening a project 
management office to aid in winning these 
projects and successfully managing them. 


Endnotes 

1. In this study, 'research' describes the spectrum of fundamental and applied scientific research 
as well as innovative technology projects. 
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2. This excludes the U.S. Department of Defense projects, for which data were unavailable. 

3. For a discussion of scientific research and serendipity, see Hackett, Parker, Conz, Rhoten, and 
Parker (2006). 

4. Project duration data and total university expenditures were available for 48 of 58 projects. 
The data gaps for cost were for NASA projects; the agency told us that the university contract 
was above $50 million for the list of projects they provided, but they did not provide us with the 
exact amount. We were unable to locate this information in the public domain for 10 projects. If 
data were available for the missing 10 projects, it would likely increase the median cost because 
NASA total project costs were generally higher. Note that cost expenditure and project duration 
data, where available, were provided by agency databases, except for NASA. NASA 
cost/duration data were obtained from govbudgets.com NASA edition or from project websites. 

5. These data were only available for nine projects. 
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